/Library/Application Support/SuperCollider/Extensions)last mod: 25-sep-07 sciss (This is a slightly updated version of the script from the symposium)
What is SwingOSC? Swing ... OSC ...
Why SwingOSC?
Requirements
/Library/Application Support/SuperCollider/Extensions)
The first step is launch the SwingOSC server. The plain way to do it, is to
/Applications/Utilities/Terminal.app, on GNOME: in one of the starter menus, on Windows: cmd.exe)cd into the SwingOSC installation directory, and launch it with the java tool--help, you will see the possible commandline options:
OSC comes in two common "transport flavours" :
Using the -u option will use UDP, while -t selects TCP. The <portNum> argument is necessary because there may be different UDP or TCP services per computer, so they are distinguished by their port (an integer number). For example, SuperCollider language uses UDP port 57120, while the audio synthesis server (scsynth) uses UDP port 57110 by default.
For simplicity, we use the ready-made shellscript SwingOSC_TCP.command (Mac OS X) or SwingOSC_TCP.sh (Linux) to use TCP on port 57111. (Sorry the .bat windows script seems to missing at the moment, but you can use the one that comes with Psycollider). This also uses the -L option which forbids access from remote computers, so you will need to remove the -L when your GUI server is supposed to be on a computer different from the one running SuperCollider!
Now let's see if the server responds. We do it in a very low level way, so it becomes more transparent what's happening behind the scenes. In SuperCollider we need to create a TCP client socket connected to the server:
n = NetAddr( "127.0.0.1", 57111 );
n.connect; // necessary for TCP, for UDP omit this line!
n.sendMsg( '/print', '[', '/local', \userName, '[', '/method', 'java.lang.System', \getProperty, 'user.name', ']', ']');
The last line looks a bit complicated, especially if you are not familiar with the java API. Don't worry, the low-level communication will disappear under the hood in a second. Just check that the your user name is correctly printed in the terminal.
Now let's see what we can do with the existing GUI classes of SuperCollider:
g = SwingOSC.default;
g.connect; // only necessary when you start SwingOSC without the -h option, so leave it away when using the .command or .sh shell scripts
JSCWindow.viewPalette;
The left window is the default look on Mac OS X. The right window was produced by changing the look-and-feel first, using this line:
g.sendMsg( '/method', 'javax.swing.UIManager', \setLookAndFeel, "com.sun.java.swing.plaf.motif.MotifLookAndFeel" );
JSCWindow.viewPalette;
Note: to return to aqua look-and-feel (on Mac), use
"apple.laf.AquaLookAndFeel".
Some of the standard gadgets are rendered using the look-and-feel, as you see, for example the JSCSlider, the JSCPopUpMenu etc. Others – like JSC2DSlider or JSCButton – have been customly written for SwingOSC and look the same on all platforms and with all look-and-feels. (see also: www.javootoo.com)
Note: the variable
gnow holds the instance of the default SwingOSC server. TheSwingOSCclass is modelled after theServerclass (the client representation of scsynth). The methodsendMsgsends an OSC message to the server.
Note: SuperCollider only knows four types of OSC arguments: Strings (s), Integers (i), Floats (f), and Blobs (b). The mixed use of Strings ("com.java...") and Symbols ('/method',\setLookAndFeel) is a mere question of taste here.
The OSC command here follows the pattern'/method', <objectID or className>, <methodName> ... <methodArgs>
See also the file OSC-Command-Reference.html in the SwingOSC folder.
For the above call, consult also the API documentation: java.sun.com/j2se/1.4.2/docs/api/javax/swing/UIManager.html
While you can talk directly (low-level) to GUI java classes with SwingOSC, e.g. instantiate a javax.JFrame and inside a javax.JButton, we are going to use high-level classes in SuperCollider which are more or less closely linked to java counterparts on the server. So at the moment we will forget about the details of the java world.
The starting point for every GUI is a window:
w = JSCWindow.new; // this creates the window (it's still invisible)
w.front; // this makes the window actually visible
The window implies a container view (it's the socalled JSCTopView) which can be filled with child components.
Note: I'm going to use the term 'component' synonymously with 'view', sometimes 'gadget' (where 'component' is more general as it can be another container view, and 'gadget' sounds more like it's a button or slider etc.)
Child components can be buttons, sliders, popup-menus, envelope-views etc. Each component is created with a Rect argument to specify its bounds inside the parent view (the window), and gets automatically added to the parent view:
// creates and adds a 2D-Slider inside the window w:
x = JSC2DSlider.new( w, Rect( 10, 10, 160, 160 ));
// whenever the window was already made visible, we need to refresh it to
// actually show the newly added component:
x.refresh;
The user can now interact with the GUI, but we need a means to be notified about its actions. Most gadgets allow you to assign an action-function that gets called whenever the user modifies the gadget's state (e.g. drags the slider in the example above):
(
x.action = { arg view; // the argument to the action function is the component
("The slider's value is now " ++
view.x.round( 0.01 ) ++ " / " ++
view.y.round( 0.01 )).postln;
};
)
There are more specialized action functions that can be assigned: Actions for keyboard typing (keyDownAction and keyUpAction), actions for mouse control (mouseDownAction, mouseUpAction, mouseOverAction, mouseDragAction), actions for handling drag-and-drop (canReceiveDragHandler, beginDragAction, receiveDragHandler), an action when the component is removed (onClose). Here is an example:
// first create a second slider component
y = JSC2DSlider.new( w, Rect( 200, 10, 160, 160 )); y.refresh;
// a copy+paste logic: pressing 'c' copies the x and y value, 'v' pastes
// the values (try to copy from the left to the newly created right view!)
(
var clipboard, func;
func = { arg view, char, modifiers, unicode, keycode;
var handled;
("Pressed char is '" ++ char ++ "'").postln;
switch( char,
$c, {
"Copy!".postln;
clipboard = view.x @ view.y;
handled = true;
},
$v, {
if( clipboard.notNil, {
"Paste!".postln;
view.x = clipboard.x;
view.y = clipboard.y;
});
handled = true;
});
// if the result of the keyDownAction is not nil,
// the key press is 'consumed' (not processed by any
// of the component's parent views)
handled;
};
x.keyDownAction = func;
y.keyDownAction = func;
)
Another example for mouse control:
( // Colorize the view's background while dragging the mouse [ x, Color.red, y, Color.blue ].pairsDo({ arg view, color; view.mouseDownAction = { arg view, x, y, modifiers, buttonNumber, clickCount; view.background = color; }; view.mouseUpAction = { arg view, x, y, modifiers, buttonNumber, clickCount; view.background = Color.clear; }; }); )
If you are not planning to mix your GUI with custom java components, are merely relying on the ready-made component classes that come with SwingOSC, and you are giving away your code to other people, it is highly recommended to make an abstraction from the actual component classes (such as JSCWindow, JSC2DSlider, etc.).
Instead you use a special factory class called GUI. Using this class, your GUI code can be rendered with other GUI libaries, not just SwingOSC. For example, on Mac OS X, you can choose to present the GUI using the original Cocoa GUI classes, and some basic classes already exists for an Emacs integrated GUI.
Using GUI is straightforward: To create a window, instead of JSCWindow.new you write GUI.window.new. To create a 2D-Slider, instead of JSC2DSlider.new, you write GUI.slider2D.new. The names of the components can be looked up the GUI help file. Here is code from above in the abstracted version; we render it twice with SwingOSC and Cocoa GUI-Kits (the latter only works on Mac OS X!):
( [ \swing, \cocoa ].do({ arg name, i; GUI.useID( name, { w = GUI.window.new( name.asString, Rect( 200 + (i * 440), 200, 400, 200 ), false ); 2.do({ arg j; GUI.slider2D.new( w, Rect( 10 + (j * 200), 10, 160, 160 ))}); w.front; })}); )
SuperCollider comes with a bunch of useful built-in visualizations and GUI-controls. They are accessed by calling special methods on objects that can be visualized (such as an array of numbers) or controlled by a GUI. They use the current GUI kit which can be switched using GUI.swing or GUI.cocoa. For example, every object can be "inspected" (all its fields are shown, those with setters can be modified):
GUI.swing; // or GUI.cocoa if you like
Server.default.options.inspect;
The inspector shows the current field values using a JSCDragSource (for read-only fields) or a JSCDragBoth (for read-and-write fields):
You can thus modify the fields with simple drag-and-dropping. Here is a window to select a sampling rate from:
(
var rates = [ 44100, 48000, 88200, 96000 ], dragSource;
w = GUI.window.new( "SR", Rect( 600, 300, 128, 72 ), false );
dragSource = GUI.dragSource.new( w, Rect( 4, 34, 120, 26 ))
.object_( rates.first );
GUI.popUpMenu.new( w, Rect( 4, 4, 120, 26 ))
.canFocus_( false ) // disable ugly focus border, we don't need it
.items_( rates.collect( _.asString ))
.action_({ arg view; dragSource.object = rates[ view.value ]});
w.front;
)
Another useful "plusGUI" is browse which can be called on any class:
JSCDragView.browse; // show the class browser for JSCDragView
To visualize data, plot and scope can be used. plot works "offline" and can be called on an Array, Signal, Buffer or Env object. scope is a realtime tool and can be called on a UGen-Graph-Function, a Server or a Bus:
// 1000 samples from the cauchy distribution centered around 0.0
Array.fill( 1000, { 0.cauchy }).plot;
// a basic envelope
Env.linen( attackTime: 0.1 ).plot;
// microphone input signal
s.waitForBoot({ Bus( \audio, s.options.numOutputBusChannels, 1 ).scope })
// some synth
s.waitForBoot({{ Saw.ar( mul: 0.25 )}.scope })
Sometimes the ready-made components that come with SwingOSC are not sufficient for your GUI demands. In this case, you have two options: either you develop a custom Java (Swing) component – something we will be looking at in chapter IX –, or (a bit easier) you develop a custom component in SuperCollider, using the JSCUserView class. A JSCUserView at first is a very plain thing. The actual component rendering is performed by assigning a drawFunc function which utilizes the special JPen class. JPen contains methods for painting basic shapes such as lines, rectangles, circles etc. Here is a simple peak meter view:
(
// we store the current GUI and it's pen class (e.g. JPen)
// in a variable because they might change while the component
// exists and would thus produce an error when the Swing
// user view tries to render using the cocoa Pen...
var gui = GUI.current, pen = gui.pen, pp = 0,
numSegments = 8, decibelsPerSegment = 4.5, colors,
synth, resp;
colors = Array.fill( numSegments, { arg i;
Color.hsv( i / numSegments * 0.5, 1.0, 0.5 );
});
w = gui.window.new( "Meter", Rect( 200, 200, 128, 200 ));
w.view.background = Color.black;
v = gui.userView.new( w, Rect( 44, 4, 40, 192 ))
.canFocus_( false ) // so we don't see the focus border
.resize_( 4 ) // the view grows vertically when the window is resized!
.drawFunc_({ arg view; var bounds, peakSeg;
// view.bounds returns the rectangle bounds of the view
// relative to the top left corner of its window
bounds = view.bounds;
// to simplify drawing we shift and scale the coordinate system
pen.translate( bounds.left, bounds.top );
pen.scale( bounds.width, bounds.height );
peakSeg = (pp.ampdb.neg / decibelsPerSegment).clip( 0, numSegments ).asInteger;
if( peakSeg < numSegments, {
(peakSeg .. (numSegments-1)).do({ arg i;
pen.fillColor = colors[ i ];
pen.fillRect( Rect( 0, i / numSegments, 1, 0.8 / numSegments ));
});
});
});
s.waitForBoot({
synth = { var inp, peakPeak, trig;
inp = AudioIn.ar( 1 );
trig = Impulse.kr( 20 );
peakPeak = RunningMax.ar( inp, trig ) - RunningMin.ar( inp, trig );
SendTrig.kr( trig, 0, peakPeak );
}.play;
resp = OSCpathResponder( s.addr, [ '/tr', synth.nodeID ], { arg time, resp, msg;
pp = msg[ 3 ];
{ v.refresh }.defer;
}).add;
});
// a function that get's called when the window is closed:
// stop the metering synthesizer
w.onClose = { synth.free; resp.remove };
w.front;
)
Note: the view is repainted using v.refresh. This is placed inside a { }.defer block in order to make it compatible with cocoa GUI. While swing GUI doesn't have that restriction, in cocoa GUI (Mac OS X native) methods on components can only be called inside the AppClock thread. { }.defer makes sure its body is executed on that thread.
When designing a GUI, there is a useful pattern that we can follow. It is called MCV = Model-Controller-View because it divides the interactivity process into these three parts:
(public domain via en.wikipedia.org)
The idea is that we have some object that can be manipulated, the model. The model is visually presented by the view and manipulated by the view or any other controller (such as evaluating text in SC, or MIDI input etc.). The crucial point is that the model doesn't know about the view, hence the user interface can be changed or omitted later without destroying the code or loosing functionality.
My suggested way of implementing a MCV like structure in SC is to use a very basic mechanism that is built into every Object: Dependant-registration. It works like this:
~model = Dictionary.new;
~ctrlSet = { arg key, value; ~model.put( key, value ); ~model.changed( key, value )};
w = GUI.window.new.front;
~viewA = GUI.slider.new( w, Rect( 4, 4, 380, 26 )); w.refresh;
~viewB = GUI.slider.new( w, Rect( 4, 34, 380, 26 )); w.refresh;
~ctrlA1 = { arg view; ~ctrlSet.value( \a, view.value )};
~viewA.action = ~ctrlA1;
~viewA.onClose = { ~ctrlA2.remove };
~ctrlA2 = Updater( ~model, { arg obj, key, val; if( key === \a, {{ ~viewA.value = val }.defer })});
~ctrlB1 = { arg view; ~ctrlSet.value( \b, view.value )};
~viewB.action = ~ctrlB1;
~viewB.onClose = { ~ctrlB2.remove };
~ctrlB2 = Updater( ~model, { arg obj, key, val; if( key === \b, {{ ~viewB.value = val }.defer })});
The Updater class calls addDependant on the model. The model keeps a list of dependants. When the model's changed method is called, all dependants are notified about the change and can act accordingly. This way we can add logic that operates on the model without having to know about all the dependants (i.e. view or the controller for the view):
~rout = fork { inf.do({ ~ctrlSet.value( \a, ((~model[ \a ] ? 0) + 0.1.bilinrand).wrap( 0 ,1 )); 0.1.wait })};
~rout.stop;
(
s.waitForBoot({
~synth = { var inp, peakPeak, trig;
inp = AudioIn.ar( 1 );
trig = Impulse.kr( 20 );
peakPeak = RunningMax.ar( inp, trig ) - RunningMin.ar( inp, trig );
SendTrig.kr( trig, 0, peakPeak );
}.play;
~resp = OSCpathResponder( s.addr, [ '/tr', ~synth.nodeID ], { arg time, resp, msg;
~ctrlSet.value( \b, msg[ 3 ].clip( 0, 1 ));
}).add;
});
)
// the model continues to work without the GUI:
w.close;
~bang = Updater( ~model, { arg obj, key, val; if( key === \b and: { val > 0.5 }, { "Bang!".postln })});
~resp.remove;
~bang.remove;
If you wish to integrate other java gadgets for which no implementations exists in SuperCollider, there is two approaches: The first one is fast and well suited for presentation-gadgets. Using the JSCPlugView class, you can easily add new components to a window. The limitation here is the missing automatic invocation of action functions. The second approach is to write a proper subclass of JSCView. Often you can use the first approach to prototype that view.
For example, we might want to have a JSpinner component. The functionality of JSpinner is similar to JComboBox (aka JSCPopUpMenu), but it doesn't show a popup menu, instead an up and down arrow allow the user to step through the possible items. A spinner looks like this:
SwingOSC can be used to rather easily script the java language. That is, we can create an manipulate java objects through a proxy on the SuperCollider client side, using the JavaObject class:
// create an instance of java.awt.Frame
~jframe = JavaObject( "java.awt.Frame" );
// all method calls to the object get forwarded to the
// server who tries to find the appropriate java method to call...
~jframe.setSize( 200, 300 );
~jframe.setTitle( "Schnuck" );
~jframe.setVisible( true );
// when we are done, we should destroy the object reference
// on the server to allow garbage collection
~jframe.dispose; // this is a method in java.awt.Frame! the object still exists!
~jframe.destroy; // this deletes the object reference
// to return primitive values to SC, append an
// underscore to the method name. Warning: since the
// communication with OSC cannot be performed inplace,
// the method call must be wrapped into a Routine (that's what 'fork' does)!
//
// Example: create an instance of java.util.Random
~jrand = JavaObject( "java.util.Random" );
// query a new random value
fork { ~jrand.nextFloat_.postln }
JSCPlugView simply takes an existing JavaObject and wraps it in a handler that is compatible with JSCView, so you can use it in the regular GUIs:
~spinListModel = JavaObject( "javax.swing.SpinnerListModel" );
~spinListModel.setList( List[ "Apple", "Pear", "Banana", "Mango" ]);
~spin = JavaObject( "javax.swing.JSpinner", nil, ~spinListModel );
w = JSCWindow.new.front;
JSCPlugView( w, Rect( 4, 4, 200, 30 ), ~spin );
~spin.setValue( "Mango" );
Using the underscore style, you can query the currently selected value:
fork { ~spin.getValue_.postln };
... but we would rather want to be automatically informed about user actions. We have solved this problem by writing a JSCView subclass that creates an instance of de.sciss.swingosc.ChangeResponder, a helper class that attaches itself to the view and when the user modfies the value, the change is forwarded to SuperCollider via OSC. The ChangeResponder ist created like this:
JavaObject( "de.sciss.swingosc.ChangeResponder", this.server, this.id, \value )
this.id which is the reference to the JSpinner to listen to
\value which is the property to query and send back upon user action. So, when the user manipulates the spinner, getValue is called and the result sent back by the ChangeResponder using an OSC-Message [ '/change', <spinnerID>, \performed, \value, <currentValue> ]
Here is the full class:
// SIMPLE TEST CLASS FOR DEN HAAG SYMPOSIUM !
JSCSpinnerDenHaag : JSCView {
var <items, <value = 0;
var acResp; // OSCpathResponder for change listening
var model; // JavaObject of javax.swing.SpinnerListModel
var changeResp; // JavaObject of de.sciss.swingosc.ChangeResponder
var spin; // JavaObject of javax.swing.JSpinner
value_ { arg val;
value = this.prFixValue( val );
if( items.size > 0, {
spin.setValue( items[ value ]);
});
}
prFixValue { arg val;
^val.clip( 0, items.size - 1 );
}
items_ { arg array;
items = array;
model.setList( items.asList );
}
prClose {
model.destroy;
changeResp.remove;
changeResp.destroy;
acResp.remove;
^super.prClose;
}
prSCViewNew {
var result;
acResp = OSCpathResponder( server.addr, [ '/change', this.id ], { arg time, resp, msg;
var newVal = items.indexOfEqual( msg[ 4 ].asString );
if( newVal.notNil and: { newVal != this.value }, {
value = newVal;
{ this.doAction }.defer;
});
}).add;
spin = JavaObject.basicNew( this.id, this.server );
model = JavaObject( "javax.swing.SpinnerListModel", this.server );
result = super.prSCViewNew([
[ '/local', this.id, '[', '/new', "javax.swing.JSpinner" ] ++ model.asSwingArg ++ [ ']' ]
]);
changeResp = JavaObject( "de.sciss.swingosc.ChangeResponder", this.server, this.id, \value );
^result;
}
}
... and here some test code:
(
w = JSCWindow.new;
x = JSCSpinnerDenHaag( w, Rect( 4, 4, 200, 30 ))
.action_({ arg view; ("Selected index is " ++ view.value ++ "; item is " ++
view.items[ view.value ]).postln });
w.front;
)
x.items = [ "Apple", "Pear", "Banana", "Mango" ];
To access classes that are not part of the Java SE and which are not in the system class path, you will need to add them to the dynamic class loader, using the addClasses method in SwingOSC. Here is an example for JFreeChart (download from sourceforge.net/projects/jfreechart):
(
// assuming you have downloaded jfreechart-1.0.6, add these two
// jars to the class path (replace the dictory with your JFreeChart
// install dir!)
x = "file:///Users/rutz/Desktop/jfreechart-1.0.6/lib/";
g.addClasses( x ++ "jfreechart-1.0.6.jar", x ++ "jcommon-1.0.10.jar" );
)
Now all classes in those two jars should be accessible via SwingOSC. We create a simple pie-chart:
(
var data, plot, gen;
data = JavaObject( "org.jfree.data.general.DefaultPieDataset" );
Dictionary[
("Burundi" -> 90),
("Ethiopia" -> 110),
("Democratic Republic of Congo" -> 110),
("Liberia" -> 110),
("Malawi" -> 160),
("Guinea-Bissau" -> 160),
("Eritrea" -> 190),
("Niger" -> 210),
("Sierra Leone" -> 210),
("Rwanda" -> 210.0)]
.keysValuesDo({ arg key, value; data.setValue( key,value )});
plot = JavaObject( "org.jfree.chart.plot.PiePlot", nil, data ); data.destroy;
gen = JavaObject( "org.jfree.chart.labels.StandardPieSectionLabelGenerator", nil, "{0} ({1})" );
plot.setLabelGenerator( gen ); gen.destroy;
~chart = JavaObject( "org.jfree.chart.JFreeChart", nil, "Ten Poorest Countries", JFont( "Helvetica", 24 ), plot, true ); plot.destroy;
)
Now display it, using org.jfree.chart.ChartPanel wrapped into a JSCPlugView:
(
w = JSCWindow( "JFreeChart", Rect( 200, 200, 560, 440 ));
JSCPlugView( w, Rect( 2, 2, 556, 396 ),
JavaObject( "org.jfree.chart.ChartPanel", nil, ~chart ))
.onClose_({ ~chart.destroy })
.resize_( 5 );
JSCStaticText( w, Rect( 2, 400, 556, 36 ))
.resize_( 8 )
.align_( \center )
.string_( "(based on 2004 GNP per capita in US$)" );
w.front;
)
The result should look similar to this:
